Displaying computer-aided detection information with associated breast tomosynthesis image information

ABSTRACT

Methods, systems, and related computer program products for processing and displaying computer-aided detection (CAD) information associated with medical breast x-ray images, such as breast x-ray tomosynthesis volumes, are described. An interactive graphical user interface for displaying a tomosynthesis data volume is described that includes a display of a two-dimensional composited image having slabbed sub-images spatially localized to marked CAD findings. Also described is a graphical navigation tool for optimized CAD-assisted viewing of the data volume, comprising a plurality of CAD indicator icons running near and along a slice ruler, each CAD indicator icon spanning a contiguous segment of the slice ruler and corresponding in depthwise position and extent to a subset of image slices spanned by the associated CAD finding, each CAD indicator icon including at least one single-slice highlighting mark indicating a respective image slice containing viewable image information corresponding to the associated CAD finding.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/676,222, filed Aug. 14, 2017, now U.S. Pat. No. 10,368,817, which isa continuation of U.S. patent application Ser. No. 14/850,442, filedSep. 10, 2015, now U.S. Pat. No. 9,763,633, which is a continuation ofU.S. patent application Ser. No. 14/037,821 filed Sep. 26, 2013, nowU.S. Pat. No. 9,146,663, which is a continuation of U.S. patentapplication Ser. No. 12/899,523, filed Oct. 6, 2010, now U.S. Pat. No.8,547,402, which claims the benefit of U.S. Provisional PatentApplication Ser. No. 61/249,311, filed Oct. 7, 2009, entitled “Deviceand Method for Displaying CAD Marks in Three Dimensional Images,” whichis incorporated by reference herein. The subject matter of one or morepreferred embodiments described herein relates to the subject matter ofthe commonly assigned U.S. Ser. No. 12/330,176, filed Dec. 8, 2008,entitled “Device and Method for Displaying Feature Marks Related toFeatures in Three Dimensional Images on Review Stations,” whichpublished on Jun. 10, 2010 as US 2010/0141654A1 and which isincorporated by reference herein.

FIELD

This patent specification relates to the computer-aided detection (CAD)of anatomical abnormalities in medical images. More particularly, thispatent specification relates to the display of CAD results fortwo-dimensional and three-dimensional image volumes, with one or morepreferred embodiments having particularly advantageous application totwo-dimensional projection x-ray mammograms of the breast and/or x-raytomosynthesis image volumes of the breast.

BACKGROUND

Progress toward all-digital medical imaging environments hassubstantially increased the speed at which large amounts of medicalimage information can be accessed and displayed to a radiologist. X-raybased imaging for breast cancer screening/diagnosis is a particularlyimportant field that is experiencing such information-expandingtechnological progress. Historically, breast cancer screening/diagnosishas been based on conventional x-ray projection mammography techniquesin which an x-ray source projects x-rays through a breast that isimmobilized by compression against a breast platform. A two-dimensionalprojection image of the breast, referred to as a mammogram, is capturedby a film or digital x-ray detector located beneath the breast platform.

Although conventional x-ray mammography is currently recognized as oneof the best FDA-approved methods for detecting early forms of breastcancer, it is still possible for cancers to be missed duringradiological viewing of the mammogram. A variety of factors, such asbreast density, may contribute to the failure to detect breast cancers.

For these and other reasons, substantial attention and technologicaldevelopment has been dedicated toward breast x-ray tomosynthesis, whichis similar in many respects to conventional x-ray mammography exceptthat the x-ray source is no longer stationary, but instead rotatesthrough a limited angle relative to the breast platform normal (e.g.,−15 degrees to +15 degrees) while several projection images (e.g., 10-15projection images) are acquired by the x-ray detector. The severalprojection images are then mathematically processed to yield a number(e.g., 40-60) of tomosynthesis reconstructed images, each correspondingto a different slice of breast tissue, which can then be examined by theradiologist. Whereas a particular cancerous lesion positioned within aregion of dense fibroglandular tissue might have been obscured in asingle conventional x-ray mammogram view, that lesion could be readilyapparent within a set of tomosynthesis reconstructed imagesrepresentative of individual slices through the dense fibroglandulartissue.

Although x-ray tomosynthesis imaging and computed tomography (CT)imaging are both generally considered to be three dimensional imagingmodalities, there are differences between these two modalities that canhave an impact on the way their associated data volumes are bestprocessed and/or reviewed for the detection of anatomical abnormalities.Concordant with the favorability of x-ray tomosynthesis imaging over CTimaging from the perspectives of radiation dose and cost/complexity ofthe image acquisition equipment, the number and angular range ofprojection images for x-ray tomosynthesis imaging is less than for CTimaging, which requires a minimum angular span of at least 180 degreesplus a fan beam angle. However, unlike with CT imaging, x-raytomosynthesis imaging is not capable of providing a “true” value for thex-ray absorption property of any particular point in the imaged volume,but instead only provides such value in inseparable combination withvaryingly “blurred” versions of the absorption property from other partsof the imaged volume. The number of distinct reconstructed image slicescontaining useful and anatomically differentiating information issubstantially less for x-ray tomosynthesis than for CT imaging, andx-ray tomosynthesis reconstructed images are often artifact-laden andhighly anisotropic according to the particular range and orientation ofthe tomosynthesis imaging arc traversed. For these reasons, x-raytomosynthesis imaging is sometimes termed a “quasi” three-dimensionalimaging modality, with CT imaging representing a “true”three-dimensional imaging modality. While the preferred embodimentsinfra are particularly advantageous when applied to the peculiarities ofx-ray tomosynthesis image volumes, it is nevertheless to be appreciatedthat one or more aspects of the described embodiments can be extended tothe processing and display of CT data volumes without departing from thescope of the present teachings.

Computer-aided detection (CAD) refers to the use of computers to analyzemedical images to detect anatomical abnormalities therein, and/or theuse of computers to otherwise process image information in a manner thatfacilitates perception of the medical image information by aradiologist. Sometimes used interchangeably with the term computer-aideddetection are the terms computer-aided diagnosis, computer-assisteddiagnosis, or computer-assisted detection. CAD findings are most oftencommunicated in the form of annotation maps comprising graphicalannotations (CAD markers) overlaid on a diagnostic-quality orreduced-resolution version of the medical image. Substantial effort andattention has been directed to increasing the analysis capabilities ofCAD systems, resulting in ever-increasing amounts of information that isavailable to the radiologist for review. Thousands of CAD systems forconventional x-ray mammography are now installed worldwide, and are usedto assist radiologists in the interpretation of millions of mammogramsper year.

Development and commercialization of CAD systems capable of identifyinganatomical abnormalities in x-ray tomosynthesis data volumes alsocontinues. However, in progressing from conventional x-ray mammographyto breast x-ray tomosynthesis imaging, practical issues arise withregard to the rising volume of data requiring review by the radiologist.Whereas there are usually just four conventional x-ray mammogram imagesper patient, there can be hundreds of tomosynthesis reconstructed imageslices (e.g., 40-60 slices for each of the four views). As more visualinformation becomes available, an important challenge is to present suchinformation to the radiologist effectively and efficiently such thatscreening for abnormalities can be done thoroughly and effectively, andyet in a reasonable time to be practical, and diagnostic assessment canalso be facilitated.

Of particular importance is the manner in which an image reviewworkstation displays CAD markers to the radiologist in the large stackof tomosynthesis reconstructed images. For CAD markers displayed duringa first reading, it is desirable that the CAD markers not be overlyobtrusive on their corresponding image, it is also desirable that theynot be readily overlooked as the radiologist moves through his/herexamination of the image slices. For CAD markers displayed as part of asecond reading, the CAD marks may be more obviously displayed, but dueto the sheer volume of available tomosynthesis image slices, it is stillpossible that CAD markers may be overlooked. One problem that may beencountered when reviewing CAD markers in a tomosynthesis data set isthat the markers are not located on all of the image slices. In fact, ina given set it may be that CAD markers are only located on a few of theimages. One method of facilitating a more efficient CAD review during aradiological reading is described in the commonly assigned U.S. Pat. No.763,053,362, which is incorporated by reference herein, and whichdescribes a ruler identifying the slices for display. Each slice thatcontains a marker has an indicator positioned next to the ruler. Withsuch an arrangement a reviewer can quickly identify a slice with a CADmark and transition rapidly to the slice of interest by selecting themarker that is near the ruler, thereby increasing reviewing efficiency.

Another method of increasing the efficiency of CAD review duringradiological reading is described in the commonly assigned US2009/0087067A1, which is incorporated by reference herein, and whichdescribes including CAD proximity markers on one or more image slicesneighboring those that contain actual CAD markers, such that a reviewerwho is quickly paging through many slices will be less likely to missthe CAD-marked slices. Both of the above techniques reduce the chancethat an image slice will be overlooked during review, yet each stillrequire sifting through multiple images to identify those images withthe most relevant information. Other issues arise as would be readilyapparent to one skilled in the art in view of the present disclosure.

SUMMARY

Provided according to one or more preferred embodiments are methods,systems, and related computer program products for processing anddisplaying computer-aided detection (CAD) information in conjunctionwith breast x-ray tomosynthesis data volumes and/or breast x-rayprojection images. In one preferred embodiment, an interactive userinterface for displaying breast x-ray tomosynthesis information andrelated CAD information is provided. In operation, a two-dimensionalbreast x-ray tomosynthesis reconstructed image slices corresponding to arespective plurality of slice depths in a breast volume is received,along with a plurality of CAD findings associated with the breastvolume. Each CAD finding identifies a subset of the image slices spannedby a suspected anatomical abnormality, as well as locations therein ofthe suspected anatomical abnormality. The received image slices and thereceived CAD findings are processed to generate a two-dimensionalcomposited image of the breast volume, wherein the two-dimensionalcomposited image comprises (a) a first slabbed sub-image of a firstlocalized neighborhood that laterally encompasses a first of the CADfindings, the first slabbed sub-image being formed by slabbing thesubset of image slices spanned by the first CAD finding, and (b) anon-CAD-specific sub-image of at least one neighborhood of the breastnot associated with any of the CAD findings. The two-dimensionalcomposited image is then displayed on a user display. Preferably, thefirst slabbed sub-image does not include contributions from image slicesnot spanned by the first CAD finding, and the non-CAD-specific sub-imageis derived from image information other than the particular subset ofimage slices slabbed to form the slabbed sub-image.

According to another preferred embodiment, an interactive user interfacefor displaying breast x-ray tomosynthesis information and related CADinformation is provided, in which there is received the above-describedimage slices and CAD findings. Displayed on a user display is a twodimensional diagnostic image comprising either (i) a single one of thereceived image slices, or (ii) a plurality of depthwise adjacent ones ofsaid received image slices slabbed together, the diagnostic image thusbeing characterized by an image depth and an image thickness. Providedon the user display in visual proximity to the two dimensionaldiagnostic image is a graphical depth navigation tool configured tographically communicate to a user the image depth and the imagethickness of the currently displayed diagnostic image, and to allow usercontrol thereof. The graphical depth navigation tool comprises (a) aslice ruler spatially extending in a first direction representative ofsaid image depth, and (b) a slice slider icon disposed along the sliceruler at a user-controllable position corresponding to the image depth.Further displayed on the user display is a plurality of CAD indicatoricons corresponding respectively to the plurality of CAD findings, eachCAD indicator icon running near and along the slice ruler and spanning acontiguous segment thereof that corresponds in depthwise position andextent to the subset of image slices spanned by the associated CADfinding. Further displayed is at least one single-slice highlightingmark on each of the CAD indicator icons, each single-slice highlightingmark being positioned on its associated CAD indicator icon at a locationindicative of the slice depth of a respective one of the subset of imageslices spanned by the associated CAD finding and containing viewableimage information corresponding to that associated CAD finding.

According to another preferred embodiment, an interactive user interfacefor displaying breast x-ray tomosynthesis information and related CADinformation is provided, in which there is received the above-describedimage slices and CAD findings, and in which there is displayed theabove-described two-dimensional diagnostic image associated with thebreast volume. Each of the image slices corresponds to a slice in thebreast volume that is generally transverse to a direction ofcompression. Further displayed on the user display in visual proximityto the two dimensional diagnostic image is a graphical navigation toolconfigured to graphically communicate to a user the image depth andimage thickness of the currently displayed diagnostic image, and toallow user control thereof. The graphical navigation tool comprises atwo-dimensional outline image in miniaturized form of the breast volumeas projected onto a plane along the direction of compression, thetwo-dimensional outline image having a depth dimension corresponding tothe direction of compression and a lateral dimension normal to saiddepth dimension. The graphical navigation tool further comprises a sliceslider bar extending across at least a portion of the outline image in adirection parallel to the lateral dimension, the slice slider bar havinga user-controllable position in the depth dimension that corresponds tothe image depth of the currently displayed diagnostic image. Furtherdisplayed on the user display is a plurality of CAD indicator iconscorresponding respectively to the plurality of CAD findings, each CADindicator icon being positioned on the outline image at a locationrepresentative of the location of the associated CAD finding in thebreast volume. Each CAD indicator icon has a position and extent in thedepth dimension that corresponds to the slice depths of the image slicesspanned by the associated CAD finding.

Provided according to another preferred embodiment is a method forprocessing and displaying information related to a plurality of breastx-ray tomosynthesis cases, each case comprising at least one breastx-ray tomosynthesis data volume associated with at least one breast of apatient. For each case, a target count “N” representing a target numberof marked CAD findings to be displayed to a user on a review workstationin conjunction with the at least one data volume is determined.Importantly, the target count N is independent of any breast tissueimage information contained in any of the data volumes for any of thecases. For each case, a set of candidate CAD findings is received, eachcandidate CAD finding being associated with a potentially suspiciouslesion in the at least one breast as identified by a CAD algorithm andcharacterizing the potentially suspicious lesion by a plurality ofcomputed features including a certainty of finding metric. For eachcase, up to the target count N of the candidate CAD findings aredesignated as being marked CAD findings according to the steps of (a)designating all of the candidate CAD findings as marked CAD findings ifthe number of candidate CAD findings in the received set is less than orequal to the target count N, and (b) if the number of candidate CADfindings is greater than the target count N, processing the candidateCAD findings according to their computed features to designate exactly Nof them as marked CAD findings. For each of the cases, the at least onedata volume is displayed to the user on the review workstation withviewable annotation markers thereon corresponding to each of the markedCAD findings, the review workstation not displaying annotation markerscorresponding to the candidate CAD findings that are not marked CADfindings. The user experience is thereby weighted more towardconsistency in the number of marked CAD findings per case and lesstoward uniform evaluation of the candidate CAD findings across differentcases.

Provided according to another preferred embodiment is a method forprocessing and displaying information related to a plurality of breastx-ray tomosynthesis data volumes, the method being analogous to theabove-described method except that there is specified a target count “N”representing a target number of marked CAD findings to be displayed perdata volume, regardless of the case membership of that data volume. Foreach data volume, up to the target count N of the candidate CAD findingsare designated as being marked CAD findings according to the steps of(a) designating all of the candidate CAD findings as marked CAD findingsif the number of candidate CAD findings in the received set is less thanor equal to the target count N, and (b) if the number of candidate CADfindings is greater than the target count N, processing the candidateCAD findings according to their computed features to designate exactly Nof them as marked CAD findings. The user experience is thereby weightedmore toward consistency in the number of marked CAD findings per datavolume and less toward uniform evaluation of the candidate CAD findingsacross different data volumes.

Provided according to another preferred embodiment is a method forprocessing and displaying CAD findings associated with breast x-rayimages in a manner that is at least partially dependent onfibroglandular density characteristics of the breast. A medical x-rayimage of a breast is received along with a set of candidate CAD findingsassociated with the medical x-ray image, each candidate CAD findingidentifying a location of a potentially suspicious lesion in the breastand characterizing the potentially suspicious lesion by a plurality ofcomputed features including a certainty of finding metric. There is thencomputed a fibroglandular tissue density map of the breast that is basedat least in part on information associated with the medical x-ray image.The fibroglandular tissue density map characterizes each locationtherein by a fibroglandular tissue density metric representative of anabsolute proportion, by volume, of fibroglandular breast tissue in alocal neighborhood of that location. Each of the candidate CAD findingsis then designated as being either a marked CAD finding or a non-markedCAD finding based on its associated certainty of finding metric and thefibroglandular tissue density metric at the location thereof.Preferably, in order to be designated as marked CAD findings, candidateCAD findings at locations of higher fibroglandular tissue densityrequire higher certainties of finding than is required for candidate CADfindings at locations of lower fibroglandular tissue density. Themedical x-ray image is then displayed to a user on a review workstationwith viewable annotation markers thereon corresponding to each of themarked CAD findings, the review workstation not displaying annotationmarkers corresponding to the non-marked CAD findings.

Provided according to another preferred embodiment is another method forprocessing and displaying CAD findings in a manner that is at leastpartially dependent on breast fibroglandular density characteristics,the method comprising receiving the medical x-ray image and CAD findingsas described above, and computing the fibroglandular tissue density mapas described above. The fibroglandular tissue density map is thenprocessed to detect a contiguous region of the breast therein that ischaracterized by (i) a fibroglandular tissue density metric that ishigher than a predetermined statistical threshold, and (ii) a size andshape that is sufficient to substantially obscure an anatomicalabnormality among the high fibroglandular density tissue therewithin.All CAD findings located within the detected contiguous region are thendesignated as unmarked CAD findings, regardless of their computedfeatures. The medical x-ray image is then displayed to a user on areview workstation with viewable annotation markers thereoncorresponding to each of the marked CAD findings, the review workstationnot displaying annotation markers corresponding to the non-marked CADfindings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a conceptual diagram of a breast x-ray imagingenvironment including a review workstation for interactive display ofbreast x-ray information and related computer-aided detection (CAD)information according to a preferred embodiment;

FIG. 2 illustrates a breast as compressed between two compression platesfor breast x-ray tomosynthesis image acquisition;

FIG. 3 a block diagram of a system in which user interface toolsaccording to one or more preferred embodiments may be used to manage thedisplay of breast x-ray image information;

FIG. 4 illustrates a block diagram of a user display according to apreferred embodiment including exemplary positioning of certain userinterface tools thereof;

FIG. 5 illustrates an interactive user interface display according to apreferred embodiment;

FIG. 6 illustrates a graphical depth navigation tool and CAD-assistedimage navigation therewith according to a preferred embodiment;

FIGS. 7A-7B illustrate a graphical depth navigation tool according to apreferred embodiment;

FIG. 8A illustrates an interactive user interface display according to apreferred embodiment;

FIG. 8B illustrates a navigation-assisting CAD annotation road mapwindow of the user interface display of FIG. 8A;

FIGS. 8C-8D illustrate auxiliary view windows of the user interfacedisplay of FIG. 8A;

FIG. 9 illustrates an interactive user interface display including ananatomic outlining tool according to a preferred embodiment;

FIGS. 10A-10B each illustrate an interactive user interface displayincluding a two-dimensional composited image having slabbed sub-imagesspatially localized to marked CAD findings according to a preferredembodiment;

FIG. 11 illustrates selecting candidate CAD findings associated withplural x-ray tomosynthesis cases (or plural individual x-raytomosynthesis data volumes) to mark for display according to a preferredembodiment;

FIG. 12 illustrates a fibroglandular tissue density map and associatedcertainty-of-finding thresholds for designating candidate CAD findingsas marked CAD findings according to a preferred embodiment;

FIG. 13 illustrates an interactive user interface display including aCAD validity warning and an alternative modality recommendationtriggered by an identification of a region of excessive localizedfibroglandular tissue density according to a preferred embodiment; and

FIGS. 14A-14B illustrate a conceptual diagram of forming athree-dimensional fibroglandular tissue density map from a plurality ofx-ray tomosynthesis projection images according to a preferredembodiment.

DETAILED DESCRIPTION

In describing preferred embodiments illustrated in the drawings,specific terminology is employed for the sake of clarity. However, thedisclosure of this patent specification is not intended to be limited tothe specific terminology so selected and it is to be understood thateach specific element includes all technical equivalents that operate ina similar manner.

Although the following description refers to user interfaces of one ormore preferred embodiments to facilitate dynamic review of breast x-raytomosynthesis data (either during a first or second reading of the imagedata) it will readily be appreciated by one of skill in the art that oneor more concepts of the preferred embodiments may be extended for use inviewing CAD information available in any dimension of athree-dimensional data set provided by any means, and ‘displayed’ in anymanner. Thus the below description should be viewed only as illustrativeand not limiting. Although certain terms and definitions will beprovided which have particular relevance to breast imaging it will beappreciated that equivalent elements are found in the related arts. Forexample, although mention may be made to mammograms and tomosynthesisprojection images, such images should be viewed as equivalents to anytwo dimensional image as a part of a three dimensional volume.

That said, the following abbreviations shall have the followingdefinitions throughout this application. The notation Mp refers to aconventional x-ray mammogram, which is a two-dimensional projectionimage of a breast and encompasses both a digital image as acquired by aflat panel detector or another imaging device and the image afterconventional processing to prepare it for display to a healthprofessional or for storage, such as in a PACS system (Picture Archivingand Communication System) of a hospital or another institution. Thenotation Tp refers to an x-ray tomosynthesis projection image that issimilarly two-dimensional but is taken at a respective tomosynthesisprojection angle between the breast and the source of the imaging x-rays(typically the focal spot of an X-ray tube), and also encompasses suchimage as acquired as well as such image after being processed fordisplay or for some other use. The notation Tr refers to a tomosynthesisreconstructed image that is computed from the images Tp according to atomosynthesis reconstruction algorithm, and represents a slice or slabof the breast as it would appear in a projection x-ray image of thatslice at any desired angle, not only at an angle used for Tp or Mpimages.

The terms Tp, Tr, and Mp also encompass information, in whatever form,that is sufficient to describe such an image for display, furtherprocessing, or storage. The images Mp, Tp and Tr typically are indigital form before being displayed, and are defined by informationidentifying properties of each pixel in a two-dimensional array ofpixels. The pixel values typically relate to respective measured orestimated or computed responses to x-rays of corresponding volumes inthe breast (voxels or columns of tissue).

FIG. 1 illustrates a conceptual diagram of a breast x-ray imagingenvironment 100 including a review workstation 120 for interactivedisplay of breast x-ray information and related computer-aided detection(CAD) information according to one or more of the preferred embodiments.Shown in FIG. 1 is a network 116, which may be a HIS/RIS (HospitalInformation System/Radiology Information System) network, to which iscoupled a digital mammogram acquisition device 102, a film-screenmammogram acquisition device 104, a tomosynthesis acquisition device106, and a generalized “other” medical imaging device 110 representativeof, for example, magnetic resonance imaging (MRI) acquisition devicesand ultrasound acquisition devices.

With reference to FIG. 1, a computer-aided detection (CAD) processor 112coupled to the network 116 receives digital medical images from one ormore of the devices 102, 106, and 110, and/or from a digitizer 115 thatdigitizes x-ray mammogram films 114 generated by the film mammogramacquisition device 104. For tomosynthesis data sets, an additionaltomosynthesis reconstruction processor (not shown) can be coupled to thenetwork 116 to generate and provide a plurality of tomosynthesisreconstructed image slices from x-ray tomosynthesis projection imagesprovided by the tomosynthesis acquisition device 106. Alternatively, theadditional tomosynthesis reconstruction processor can be included withor integrated into the tomosynthesis acquisition device 106. The CADprocessor 112 processes the medical images according to one or more CADalgorithms and provides CAD findings associated therewith, eachcandidate CAD finding identifying a location of a potentially suspiciouslesion in the breast and characterizing the potentially suspiciouslesion by a plurality of computed features. For an x-ray tomosynthesisdata volume, which comprise plurality of two-dimensional breast x-raytomosynthesis reconstructed image slices corresponding to a respectiveplurality of slice depths in the breast volume, each CAD findingidentifies the particular subset of the image slices spanned by asuspected anatomical abnormality, such as a potentially suspiciousmicrocalcification cluster or a potentially suspicious mass lesion,along with the locations in each slice occupied by the suspectedanatomical abnormality.

A graphical user interface implemented at a review workstation 120displays the medical images to a viewer in accordance with one or moreuser interface programs carried out on a user interface processor 128,and further provides an interactive graphical user interface fordisplaying the CAD detection information in conjunction with theassociated medical images in accordance with one or more of thepreferred embodiments described further infra. Review workstation 120comprises an interactive user interface 130 including a diagnosticdisplay 122, an administrative display 124, and user input devices 126(e.g., keyboard, mouse, trackball, pointers, etc) that are under thecontrol of the user interface processor 128. Administrative display 124is used for input and output of a wide variety of information that maybe associated with a particular set of medical images (e.g., listings,tables, plots, text descriptions, etc), as well as for systeminstallation, maintenance, updating, and related tasks. Often providedon the diagnostic display 122 at any particular time during case reviewby a radiologist is a two dimensional diagnostic image 132, variousimplementations of which are described further infra, and one or moregraphical viewing and/or navigation assistance tools 136, variousimplementations of which are also described further infra. Withparticular regard to the user input devices 126 illustrated in FIG. 1,it is to be appreciated that the user interface features describedherein are broadly applicable for a variety of user interface hardwareimplementations and that many variations are within the scope of thepreferred embodiments. By way of example, whereas at least one exampleherein illustrates paging commands as being mouse clicks, any user inputthat instantiates a like paging operation (ranging from keystrokesequences to touchscreen inputs to virtual reality glove movement) iswithin the scope of the preferred embodiments.

Preferably, the various medical images and related information arecommunicated according to the DICOM (Digital Imaging and Communicationsin Medicine) standard and the network 110 supports the TCP/IP protocol,which is used as the transport protocol for the DICOM standard. Alsocoupled to the network 110 is a PACS archive 118, generally representinga repository for medical information associated with the medical imagingenvironment, including both current and archived images, current andarchived CAD results, radiology reports for completed cases, and soforth.

The preferred embodiments described herein are seamlessly layered uponan existing CAD workflow, in which the digital or digitized medicalimages are processed by the CAD processor 112, and in which the medicalimages and their related CAD results are subsequently displayed at thereview workstation 120 to a viewer, who makes a clinical determinationtherefrom. Although one or more of the preferred embodiments isparticularly advantageous in the context of en masse breast cancerscreening contexts, the clinical determination to be made by the viewercan be in relation to screening, diagnosis, follow-up, or any of avariety of other activities without departing from the scope of thepreferred embodiments.

Notably, the medical imaging environment of FIG. 1 is presented by wayof example only and is not intended to limit the scope of the preferredembodiments to this particular scenario. By way of example, differentcombinations of the devices of FIG. 1 can be placed adjacently to eachother or integrated into the same hardware boxes without departing fromthe scope of the preferred embodiments. By way of still further example,the network 110 can be a wide-area network with the different nodesbeing distributed throughout a city, a country, or the world.Alternatively, and by way of still further example, some or all of thetransfer of digital information can be achieved by physical transfer ofdisks, memory sticks, or other digital media devices without departingfrom the scope of the preferred embodiments. In view of the presentdisclosure, a person skilled in the art would be able to implementmethods, systems, and/or computer program products capable of achievingthe described user interfaces and processing functionalities withoutundue experimentation, using publicly available programming tools andsoftware development platforms.

FIG. 2 illustrates a conceptual diagram of breast x-ray tomosynthesisimage acquisition, which is presented to provide a backdrop fordescribing certain preferred user interface features infra in theinstant specification. A breast B is positioned between an uppercompression paddle 202 a and a lower compression paddle or platform 202b, near which is disposed an x-ray detector 204. The breast iscompressed in the direction of the lower compression paddle 202 b asshown, typically with a relatively large force such as 25 lbs. Eachbreast x-ray tomosynthesis projection image is acquired by projectingx-rays R at a respective tomosynthesis projection angle θ_(T) throughthe compressed breast B from an x-ray source (not shown) positioned onone side of the compression paddles 202 a/202 b (from the upper side ofthe compression paddles in FIG. 2) toward the x-ray detector 204 that ispositioned on an opposite side of the compression paddles 202 a/202 b,(toward the lower side of the compression paddles in FIG. 2).

FIG. 3 illustrates another expression of a three dimensional imagingsystem which may advantageously incorporate the user interface tools ofthe preferred embodiments. Although FIG. 3 illustrates components of atomosynthesis system, for which the preferred embodiments areparticularly advantageous, it is to be appreciated that the scope of thepresent teachings is not so limited and may be extended to a variety ofdifferent imaging modalities that use CAD software tools in conjunctionwith multi-dimensional image data.

Illustrated in block diagram form in FIG. 3 is an x-ray data acquisitionunit 300 that includes an x-ray source 310 imaging a breast 312. Anx-ray imager 316 such as a flat panel x-ray imager commerciallyavailable from the assignee of this patent specification generatesprojection image data that can be a mammogram Mp or a tomosynthesisprojection image Tp. X-ray source 310 is mounted for movement so thatimages Tp can be taken at different angles. X-ray imager 316 can bestationary or it can also move, preferably in synchronism with movementof x-ray source 310. Elements 310 and 316 communicate with x-ray dataacquisition control 318 that controls operations according to knownmethods. X-ray image data from imager 316 is delivered to processingunit 320. Processing unit 320 comprises reconstruction software 322,which may be stored in a computer readable medium of unit 320. Thereconstruction software processes x-ray image data into Tp and Tr imagedata, which may be stored in storage device 330 as reconstructed data331 and displayed at image display unit 350 as disclosed in the variousembodiments described herein. Processing unit 320 further includes twodimensional and/or three dimensional CAD software 324 which processesthe Tp and/or Tr data. The processing unit 320 can consist of severaldifferent physical computers, that is, the reconstruction software 322might be resident on one computer and the CAD software 324 on adifferent computer, and the user interface software 325 on a thirdcomputer. CAD systems are used to assist radiologists in theinterpretation of millions of mammograms per year. X-ray mammography CADsystems are described, for example, in U.S. Pat. Nos. 5,729,620,5,815,591, 6,014,452, 6,075,879, 6,301,378 and 6,574,357, each of whichis incorporated by reference herein. Application of CAD algorithms toone or more of tomosynthesis projection images and tomosynthesisreconstructed images has been proposed in U.S. Pat. Nos. 6,748,044 and7,218,766, each of which is incorporated by reference herein.

CAD software 324 retrieves the three dimensional reconstructed data 331from storage 330 and processes the tomosynthesis data set, generatingCAD overlay images for display over each two dimensional image slice orslab. A CAD overlay image may include one or more markers which areassociated with features of a corresponding image slice or slab that aresuggestive of a cancerous or pre-cancerous lesions. The CAD overlayimages are referred to herein as the CAD data set 332 and followinggeneration may be stored in the storage device 330 along with thereconstructed data, for later display at the workstation, or forforwarding to an external display device, for example using DICOM(Digital Imaging and Communications in Medicine) interface 370. Forexample, as described in U.S. Pat. No. 6,909,795, which is incorporatedherein by reference, the tomosynthesis CAD information of the presentinvention may fixably integrated into the pixels of a secondary imagederived from a source image, and the secondary image transferred using aDICOM Secondary Capture Image Information Object Instance (SCI-IOI),either to a viewing workstation, printer or other output device.

User interface software 325 is, in one embodiment, a software modulewhich can be loaded on any system that stores three dimensional imagedata for display. The interface may be used to select the number of“threshold” CAD marks to be displayed in addition to the functionalitydescribed below. The software module is stored in a computer readablemedium of the system, and operable when executed upon by a processor ofthe system to generate an initial display which introduces the threedimensional data set to a radiologist in a manner that facilitatesreview of the data set. The user interface software 325 includesfunctionality for identifying features that correspond to a commonregion of interest, grouping the identified features, assigning an agroup identifier to the related features, identifying an initial twodimensional image slice for display when viewing each group, andpopulating a user interface data structure with feature information forthe three dimensional data set.

According to one aspect of the preferred embodiments, the user interfacealso includes several tools for improving quality and efficiency of thereview of the three dimensional data set. These tools allow the user toeasily select different regions of interest, identify and scroll throughtwo dimensional slices and slabs associated with selected regions ofinterest and obtain enhanced views of regions of interest. With such anarrangement the efficiency and effectiveness of review is improved. Itshould be noted that although the user interface is described asperforming different functions, the functionality may be delineated sothat processing, display and/or manipulation of data may each beindependently performed by any computer that has access to the imagedata.

FIG. 4 illustrates a block diagram of a user display according to apreferred embodiment including exemplary positioning of certain userinterface tools thereof. Illustrated in FIG. 4 is a block diagram of adisplay 405 including a two dimensional diagnostic image display window410, a display toolbar 430, a graphical depth navigation tool 411(including a slice ruler 418, an extent indication region 413 and aslice slider 419), a region of interest (ROI) selection tool 415, anavigation-assisting CAD annotation road map window 420, and one or morealternate region of interest (ROI) view windows 422 and 424. The twodimensional image display area 410 is used for displaying a currenttwo-dimensional diagnostic image to the radiologist, which willgenerally be a diagnostic-quality Mp image, a Tr image, a slabbed Trimage, a cine display of Tr images or slabbed Tr images, or a composited image as described further hereinbelow. The currently displayedtwo-dimensional diagnostic image 410 may further include one or more CADmarkers 414.

According to one preferred embodiment, the size or shape of the CADmarker may be related to the size and/or shape of the detected lesion.For example, a larger marker may indicate a larger ROI, or differentlyshaped markers may be selected to distinguish masses fromcalcifications, and so forth. The display toolbar area 430 may includeselectable icons that enable the user to control the displayed image;for example, by selecting different views, adjusting image contrast,displaying patient or display information or selected user interfacetools such as the slice ruler 418, and so forth. Near the displaytoolbar area 30 may be displayed particular orientation information suchas alphanumeric slice depth and slab thickness information 416corresponding to the currently displayed diagnostic image 410.

The ROI selection tool 415, extent indication region 413, slice ruler418, and slice slider 419 can be used to interactively control andmonitor the image depth and image thickness corresponding to thecurrently displayed diagnostic image. As used herein in the context ofx-ray tomosynthesis data volumes comprising a plurality of image slicesrepresentative of a respective plurality of slices of a breast volumehaving respective image depths, slabbing refers to the integration ofmultiple adjacent image slices (i.e., image slices corresponding toadjacent slices in the breast volume) into a single two-dimensionalimage. The resultant two-dimensional image, which can be termed an imageslab, a slabbed image, or a thick image slice, is characterized by animage depth, which can be expressed as an average depth of its multiplecomponent image slices, and an image thickness, which can be expressedin terms of the number of component image slices or the depthwisespatial extent of its component image slices in the breast volume. Anyof a variety of methods for integrating the multiple adjacent imageslices into a single slabbed image can be used, such as arithmeticaveraging, maximum intensity projecting, and so forth.

In operation of the user interface provided by the display 415, the usermay engage the slice slider 419 (for example, via a mouse click or thelike) to scroll through the image slices of the tomosynthesis datavolume. In addition, a user may select one or more visual indicators inthe extent indication region to obtain one or more image slices in theROI associated with the visual indicator. Alternatively, the user mayuse the ROI selection tool 415 to display a particular image slice orslabbed image associated with a particular marked CAD finding. The ROIselection tool 415 may include a pull down menu, a clickable arrowcontrol, or any other mechanism for selecting from a set of marked CADfindings. As used herein, a marked CAD finding may alternatively betermed an ROI feature group. In one preferred embodiment, selection of aparticular marked CAD finding using the ROI selection tool 415 causes acentral slice of the set of slice images spanned by that CAD finding tobe displayed. As used herein, the set of image slices “spanned” by aparticular CAD finding is a contiguous set of image slices in thetomosynthesis data volume that are collectively occupied or “touched” bythe identified anatomical abnormality, the contiguous set ranging froman uppermost image slice to a lowermost image slice. It is not requiredthat every image slice in the spanned set contain visual evidence of theanatomical abnormality, and indeed it is quite common, as in the case ofmicrocalcification clusters, for several image slices in a spanned setnot to contain any such visual information. As the term “spanned” isused herein, the set of image slices “spanned” by a particular CADfinding can optionally include a very limited number, such as one, of“framing” or “end” image slices on the top end and bottom ends of thesubset that do not themselves contain visual indications of theanatomical abnormality.

According to one preferred embodiment, a navigation-assisting CADannotation road map window 420 is provided that includes a thumbnail orsmall-scale two dimensional image that displays all CAD markings for thedata set. The two dimensional image may comprise an Mp image, a Trimage, a slabbed Tr image that is laterally representative of thetomosynthesis data volume, i.e., representative of the slab-shapedtomosynthesis data volume as “seen” from a viewpoint distal therefrom ina direction normal to the plane along which the breast is flattened. Inone embodiment the two dimensional image is displayed with all of theCAD findings for the tomosynthesis data set. In other embodiments, theCAD findings displayed in the navigation assist window are limited to aparticular number of results, or are limited by size, depth, type orother thresholding means. A reviewer may access image data associatedwith particular ROIs by selecting the CAD mark displayed in thenavigation-assisting window 420 with which it is associated. When a CADmark from the navigation-assisting window 420 is selected, the displayis updated so that the central slice of the Tr slab associated with theCAD mark is displayed in area 410. The selected CAD mark is highlightedin the diagnostic image area 410, and the slice slider and extend markerinformation for the slab is provided in the area 411. It should be notedthat although the navigation-assisting window 420 is shown as athumbnail view within the same display as diagnostic image 410, it isnot required that the image be so located or sized. While it is believedthat it is desirable to place such a navigation window near the breastimage for ease of review, it is to be appreciated that other interfacesmay place such a window elsewhere on the display, or on an alternatedisplay; such arrangements are considered to be alternate embodimentswithin the scope of the present teachings.

The alternate ROI view windows 422 and 424 provide enhanced images ofthe selected ROI. For example, in one embodiment one ROI window mayprovide a magnified view of the calcification, while another windowprovides a cluster view of the lesion. In still other preferredembodiments it is envisioned that an alternate ROI view window maydisplay a correlated portion of an historical Mp image, to enable thereviewer to gauge any degree of change for that particular region ofinterest. As with the placement of the navigation-assisting window 420,although the alternate view windows 422 and 424 are shown as thumbnailviews proximate to the breast image, other placements and sizing of thewindows are envisioned by the present preferred embodiments.

Marked CAD findings (ROIs) may also be navigated via the extentindication region 413. As described below, the extent indication regionincludes a visual indicator identifying the location and extent of eachmarked CAD finding (ROI) in the data volume. The user interface may beconfigured such that selection of a particular visual indicator causesthe central slice of that region of interest to be displayed in area410. In addition, the user interface may be configured so that selectionof a particular visual indicator (or alternatively, the selection of aCAD mark 414 in image area 10 or in navigation-assisting window 420)causes the alternate ROI view windows 422, 424 to display the region ofinterest in a magnified, cluster or other view. Also illustrated in FIG.4 is a cine control tool 477.

FIG. 4 thus displays several different methods by which a reviewer oftomosynthesis data may navigate through the data set to examine regionsof interest. It should be understood that it is not required that all ofthese navigation mechanisms be provided, but rather the preferredembodiments include interfaces which can include any combination of thedescribed tools (i.e., slice slider, ROI selector, navigation assistwindows, visual indicators, alternate view windows, etc.). Thus, thescope of the present teachings is not limited to any particularcombination of the described user interface tools.

FIG. 5 illustrates a closer view of the display 405 as populated with aparticular diagnostic display image 410 and particular examples ofinteractive user interface tools according to one or more preferredembodiments. Illustrated in FIG. 5 is an exemplary display screen whichincludes the slice ruler 418, the slice slider 419 and the ROI selectiontool 415. The slice ruler 418 includes individual hash markers 549,wherein each has marker is representative of an image slice contained inthe tomosynthesis data volume.

Illustration in the extent indication region 413 are CAD indicator icons580. As can be seen in FIG. 5, the height of each CAD indicator icon 580is matched to depthwise spatial extent of the anatomical abnormalityassociated with that CAD finding, thus providing a visual indication asto the depth of the associated lesion in terms of slices. In the exampleof FIG. 5 there are six regions of interest denoted by six CAD indicatoricons 580. Both the CAD indicator icons 580 as well as the CAD mark 414a associated with ROI #1 (CAD finding #1) are highlighted. With such anarrangement a reviewer can quickly ascertain that the lesion associatedwith CAD mark 414 a expands through the slices indicated by the CADindicator icon #1 as positioned against the slice ruler 418. In oneembodiment, the introductory image that is presented when a case isbrought up for display on the CAD system is the central slice for ROI#1, although that is not a requirement. It is to be appreciated thatother introductory images, for example those associated with the largestdetected mass or the region with the largest population ofcalcifications may be provided as introductory images, and the preferredembodiments are not limited to any particular selection of introductoryimage. Note in FIG. 5 that only CAD marks 414 a and 414 b are shown. Asthe slice slider moves up ruler 418, the additional CAD marks, which areassociated with ROIs on ‘higher’ slices, become visible. Alsoillustrated in FIG. 5 is the cine control tool 477, which includes aslidable handle for controlling the frame rate of cine playback.

Preferably, as shown in FIG. 5, each of the CAD indicator icons 580includes at least one single-slice highlighting mark 582. Eachsingle-slice highlighting mark 582 is positioned on its associated CADindicator icon 580 at a location indicative of the slice depth of arespective one of the subset of image slices spanned by the associatedCAD finding that contains viewable image information corresponding tothat associated CAD finding. Thus, for example, if the associated CADfinding is a microcalcification cluster that spans ten image slices butcontains only four small individual microcalcifications, several ofthose ten image slices will not contain any visual informationindicative of the microcalcification cluster. Advantageously, thesingle-slice highlighting marks 582 provide the user with a fast andintuitive visual description of which particular image slices willcontain viewable microcalcifications. The single-slice highlightingmarks 582 can also be used for highlighting prominent or importantindividual image slices for other types of anatomical abnormalities,such as suspicious masses. For example, for suspicious masses, there canbe provided single-slice highlighting marks 582 that represent theparticular image slices containing a “root” of a spiculation extendingfrom the mass, or alternatively the single-slice highlighting marks 582can be used to identify the image slices containing the highest-contrastcross-section of the suspicious mass. Thus, it is to be appreciated thatthe single-slice highlighting marks 582 can be used in the context of avariety of different types of CAD findings other than justmicrocalcification clusters.

Thus, illustrated in FIGS. 4-5 is a user interface display 405associated with a computer-implemented method for processing anddisplaying breast x-ray tomosynthesis information according to apreferred embodiment. A plurality of two-dimensional breast x-raytomosynthesis reconstructed image slices corresponding to a respectiveplurality of slice depths in a breast volume is received. A plurality ofCAD findings associated with the breast volume is received, each CADfinding identifying a subset of the image slices spanned by a suspectedanatomical abnormality and locations therein of the suspected anatomicalabnormality. A two dimensional diagnostic image 410 is displayed thatcomprises either (i) a single one of said received image slices, or (ii)a plurality of depthwise adjacent ones of the received image slicesslabbed together, the diagnostic image thereby being characterized by animage depth and an image thickness. Providing on the user display invisual proximity to the two dimensional diagnostic image 410 is agraphical depth navigation tool 411 configured to graphicallycommunicate to a user the image depth and the image thickness and toallow user control thereof. The graphical depth navigation toolcomprises (a) a slice ruler 418 spatially extending in a first direction(the vertical dimension in FIGS. 4-5) representative of the image depth,and (b) a slice slider icon 419 disposed along the slice ruler at auser-controllable position corresponding to the image depth. A pluralityof CAD indicator icons 580 are displayed that correspond respectively tothe plurality of CAD findings, each CAD indicator icon 580 running nearand along the slice ruler 418 of the graphical depth navigation tool 411and spanning a contiguous segment thereof that corresponds in depthwiseposition and extent to the subset of image slices spanned by theassociated CAD finding. At least one single-slice highlighting mark 582is displayed on each of the CAD indicator icons 580, each single-slicehighlighting mark 582 being positioned on its associated CAD indicatoricon 580 at a location indicative of the slice depth of a respective oneof the subset of image slices spanned by the associated CAD finding andcontaining viewable image information corresponding to that associatedCAD finding.

FIG. 6 illustrates the graphical depth navigation tool 411 and ROIselector tool 415 at successive states of CAD-assisted image navigationaccording to a preferred embodiment. Responsive to a user selection atthe ROI selector tool 415, such as by placing a mouse icon 582 thereonas shown and entering a mouse click, a next CAD finding (beginning withCAD finding #3 in FIG. 6) is established. The displayed two dimensionaldiagnostic image 410 is then modified to correspond to at least oneimage slice in the subset of image slices spanned by the currentlyselected CAD finding. The slice slider icon 419 includes a thicknessindicator 419 z that extends along the slice ruler by a distance thatcorresponds to the image thickness of the displayed two dimensionaldiagnostic image. Responsive to the user selection at the ROI selectortool 415, the slice slider icon 419 and the thickness indicator 419 zare automatically adjusted to correspond to the image depth and imagethickness of the modified two dimensional diagnostic image. Thecurrently selected CAD indicator icon 580 is visually highlightedcompared to the other CAD indicator icons, such as by making it brighterand/or wider. CAD annotation markings on the currently displayed twodimensional diagnostic image are updated to reflect the currentlyselected CAD finding. As illustrated in FIG. 6, the current location ofthe slice slider icon and the size of the thickness indicator 419 z areautomatically adjusted to correspond to the currently selected CADfinding as the user clicks through the sequence of marked CAD findings.CAD-assisted image navigation according to the preferred embodiment ofFIG. 6 has been found particularly useful and effective in facilitatingprompt yet thorough review of the marked CAD findings associated withthe tomosynthesis data volume.

According to one preferred embodiment, the modified two dimensionaldiagnostic image 410 is a slabbed image formed by slabbing the subset ofimage slices spanned by the currently selected CAD finding, the slabbedimage not including contributions from image slices not spanned by thecurrently selected CAD finding. According to another preferredembodiment, the modified two dimensional diagnostic image comprises aslabbed sub-image of a localized neighborhood that laterally encompassesthe currently selected CAD finding (see, for example, the slabbedsub-image 1020 of FIGS. 10A-10B infra), along with a non-CAD-specificsub-image (see, for example, the area 1050 of FIGS. 10A-10B infra)encompassing substantially all areas in the modified two dimensionaldiagnostic image outlying that localized neighborhood. Alternatively,there can be displayed in that localized area a cine-loop sequenceconsisting of the subset of image slices spanned by the currentlyselected CAD finding. The cine-loop sequence can be shown one imageslice at a time, or can alternatively comprise a sequence of slab imagesformed by slabbing two or more adjacent ones of the subset of imageslices spanned by the currently selected CAD finding. Thenon-CAD-specific sub-image can consist of information taken from anactual two-dimensional mammographic image of the breast, a synthetictwo-dimensional mammographic image of the breast derived from thereceived image slices, a single one of the received image slices, or aslabbed image formed by slabbing two or more of the received imageslices.

Provided in conjunction with the option of CAD-based navigation based onthe ROI selector tool 415 is a manual option in which the user candirectly manipulate the slice slider icon 419 and/or the image thicknessindicator 419 z. The user is also provided with the option of navigatingat will to any particular marked CAD finding by clicking directly on thecorresponding CAD indicator icon 580, wherein the image depth and imagethickness will be automatically adjusted according to the depth andspatial extent of the associated CAD finding.

FIGS. 7A-7B illustrate a graphical depth navigation tool 702 accordingto another preferred embodiment, which can be used alternatively to orin conjunction with the graphical depth navigation tool 411 of FIGS.5-6. Graphical depth navigation tool 702 comprises a two-dimensionaloutline image 781 in miniaturized form of the breast volume as projectedonto a plane along the direction of breast compression, thetwo-dimensional outline image 781 having a depth dimension correspondingto the direction of compression (see arrow 783 in FIG. 7A) and a lateraldimension (see arrow 784 in FIG. 7A) normal to the depth dimension. Theminiaturized two-dimensional outline image can be a strictly iconicrepresentation of the breast outline not displaying internal breasttissue information from the received image slices, as shown in FIGS.7A-7B, or alternatively can be an actual miniaturized projection image(not shown) derived from the received image slices.

Graphical depth navigation tool 702 further comprises a slice slider bar719 extending across at least a portion of the outline image 781 in adirection parallel to the lateral dimension, the slice slider bar 719having a user-controllable position in the depth dimension thatcorresponds to the image depth of the currently displayed diagnosticimage. The slice slider bar 719 has a portion 719 z with auser-controllable thickness in the depth dimension that corresponds tothe image thickness of the currently displayed diagnostic image.

According to a preferred embodiment, a plurality of CAD indicator icons780 are displayed on the outline image 781, each optionally includingone or more single-slice highlighting marks 782. Each CAD indicator icon780 is positioned on the outline image at a location representative ofthe location of the associated CAD finding in the breast volume, and hasa position and extent in the depth dimension that corresponds to theslice depths of the image slices spanned by the associated CAD finding.In a manner analogous to the CAD-based navigation described above forFIGS. 5-6, provided on the user display a graphical region of interest(ROI) selector tool 715 that allows the user to sequence through themarked CAD findings one at a time. Responsive to user selection of thenext marked CAD finding, the slice depth as reflected by the depthwiseposition of the slice slider bar 719 and the slice thickness asreflected by the thickness-indicating portion 719 z are automaticallyadjusted to correspond to the next CAD finding in the sequence. The useris also provided with the option of manual navigation by directmanipulation of the slice slider icon 719 and/or the image thicknessindicator 719 z. The user is also provided with the option of navigatingat will to any particular marked CAD finding by clicking directly on thecorresponding CAD indicator icon 780, with the image depth and imagethickness being automatically adjusted according to the depth andspatial extent of the associated CAD finding.

FIG. 8A illustrates an interactive user interface display according to apreferred embodiment, including a simultaneous display of right and leftMLO tomosynthesis data volumes a breast in windows 805 and 405,respectively. Each view window 805/405 includes its own graphical depthnavigation tool including a slice slider bar 811/411 as shown, as wellas its own ROI selector tool 815/415. A navigation-assisting window 420is shown in the upper right hand corner of the left MLO image, aclose-up view of which is shown in FIG. 8B. Included innavigation-assisting window 420 are markers 821 providing a condensed,single-view CAD annotation road map for the entire data volume. Themarkers 821, as well as actual corresponding CAD markers 414 a, 414 b,and 814 c shown on the diagnostic images 410 and 810 themselves, mayvary in shape to indicate different types of lesions, or by size, toindicate a numeric measure not necessarily linearly related to thenumeric measure. For example, some markers may be related to CADfeatures such as number of calcifications, or some type of CAD scoresuch as a measure of the prominence of a combination of featuresrepresenting a lesion obviousness, for example. A reviewer can quicklyidentify a marker 821 of particular interest and promptly and easilynavigate to the slabs and associated slices by clicking on that marker821. Such multi-dimensional visual cues and data management tools serveto increase the efficiency and effectiveness of three dimensional imagereview.

Also illustrated in FIG. 8A in the upper left hand corner of the rightMLO image are alternate ROI view windows 822 and 824, close-up views ofwhich are illustrated respectively in FIG. 8C and FIG. 8D. Alternate ROIview window 822 is a CAD finding magnification window, while alternateROI view window 824 is a graphical three-dimensional rendering of a CADfinding. In one embodiment the alternate ROI view windows 822 and 824are automatically associated with the currently selected CAD finding.Although these views are provided as thumbnails, it is anticipated thatthe user interface may be designed to include control for expanding thethumbnail view, zooming in on different areas of the thumbnail view, andso forth.

Review workflow is enhanced via the user interface tools in thefollowing manner. The CAD tools may be invoked in a variety of ways at avariety of different points in the workflow. For example, CAD may beused as a first reader or a second reader, and may be selected using aCAD option on a control keyboard or by including the CAD function in aworkflow list. The CAD algorithms execute on the data set, generatingoverlays for the slices that include the CAD markings. An introductoryCAD marked two dimensional slice may be presented in area 410, having atleast one highlighted CAD mark. Should no ROI's be identified by the CADalgorithms, an indication of such finding may be provided. Dependingupon the user interface options selected, a navigation window 420 ispresented with the CAD marked two dimensional slice. The ruler and depthview area are presented and populated (illustrating the extent of eachROI associated with a CAD mark), and the slice slider is automaticallypositioned at slice location corresponding to the displayed slice. Theuser may then easily step through the ROIs and associated slices toefficiently review the three dimensional data set.

Accordingly, a system and method has been shown and described thatenables efficient use of CAD as a first or second reader on atomosynthesis or other three-dimensional data set. The method describedabove have dealt with the ability of the user to quickly navigate to aslice/slab associated with a ROI. Sometimes it may occur that a CAD markis provided and the reader is unable to readily discern what features ofthe image caused the CAD mark to be displayed.

FIG. 9 illustrates an interactive user interface display including ananatomic outlining tool according to a preferred embodiment. Suchanatomical outline tools may be invoked by a user seeking additionalinformation regarding the operation of the CAD algorithm, and inparticular it provides an understanding of what features of the imagecaused the CAD mark to be displayed by outlining the feature in theimage and highlighting detected features or calcifications. For example,FIG. 9 illustrates a display which has invoked the anatomical outlinetool for a CAD mark 970. The anatomical outline tool, for purposes ofdisplay only, modifies pixels within the affected region, dilating andhighlighting individual calcifications, drawing an outline around thedetected feature and coloring the interior of the mark black. Such atool allows the radiologist to quickly identify the features in theimage which caused the CAD algorithm to be interested in the region. Thedescribed user interfaces tools may support the anatomical outline toolin a variety of manners. The anatomical outline tool may be ‘always on’when the slice or slab that includes the CAD mark is in the imagedisplay area 410. Alternatively, the anatomical outline tool can beautomatically invoked for each respective CAD finding that is beingsequenced through by the ROI selection tool 415 of FIG. 4 or the ROIselection tool 715 of FIG. 7. Alternatively, a ‘dead man’ switch may beprovided, wherein the anatomical outline function is only enabled when aswitch is actively depressed; releasing the switch causes the anatomicaloutline function to be disabled. Conventional methods of turning thefunction on or off may also be provided. The anatomical outline markingmay be displayed only in the diagnostic image, or alternatively featuresof the anatomical outline tool may also be displayed in the alternateROI windows 822 and 824 of FIG. 8. In one preferred embodiment, theanatomical outline tool automatically-slabs the localized area aroundthe CAD finding according to the depthwise extent of the subset of imageslices spanned by that CAD finding, and a special thickness indicator919 y is provided on the slider bar to indicate that only this localizedarea around the CAD finding is so slabbed.

The above-described embodiment of FIG. 4 illustrates as the diagnosticimage 410 an Mp image, Tp image, or slabbed Tr image as a whole.Although the user interface tools may be advantageously used with thedisplay of such two dimensional images, because the image is what isactually being screened by the radiologist, the composition anddelineation of the image (i.e., a slice, a slab, etc.) is not a trivialmatter. When user is searching for and/or evaluating micro-calcificationclusters in a tomosynthesis imaging series, the review often needs toscroll back and forth between image slices, as the entire cluster israrely visible in one slice (due to the fact that individualcalcifications are quite small compared to the size of the cluster).This can be time consuming and laborious for doctors. Slab views (wheremultiple slices are combined either by averaging, maximum intensityprojection or some other means) have been proposed as a potentialsolution to this problem. It is often difficult to figure out how manyslices to include in the slab. According to one aspect of the preferredembodiments, slabbing is performed at CAD-detected cluster boundaries.Thus the slice-extent of each CAD-detected cluster is used to decide howmany slices to slab together to optimally display the cluster to theuser. For example, referring back to FIG. 5, assume that each tick markof the ruler indicates a slice of a three dimensional image. The pointsnext to the tick mark indicate that a feature in a feature grouping waslocated at the slice. It can be seen that a slice may include featuresthat are associated with different regions of interest. The CADindicator icon 580 indicates the slice extent of the feature group (orROI group), that is, all of the slices within the slice extent arerelevant when examining the ROI, and it is therefore desirable to slaball of the two dimensional slices for optimum review of the ROI.

Various mechanisms are envisioned for slabbing the two dimensionalslices. For example the CAD tool, following execution of the CADalgorithm and generation of the marks, may make a slab image of theentire view for each CAD mark; a user would therefore review a series ofpre-generated slabbed images. In another embodiment, slabbing may bedone dynamically, in response to the selection of a region of interest.In still an alternate embodiment, the entire image area 410 comprises asynthesized image, populated with all of the CAD marks from the threedimensional reconstructed data set, with the regions of interest aroundeach CAD mark being slabbed with the appropriate slices (for thecluster) at their proper location.

FIGS. 10A-10B each illustrate an interactive user interface display 1000including a two-dimensional composited image 1010 having slabbedsub-images spatially localized to marked CAD findings according to apreferred embodiment. With reference to FIG. 10A, the two-dimensionalcomposited image 1010 comprises a localized sub-image 1020 consisting ofa slabbed version of slices 11-15 as windowed laterally around thatlocality. Similarly, a localized sub-image 1030 consists of a slabbedversion of slices 25-55 in its locality, and a localized sub-image 1040comprises a slabbed version of slices 1-20 in its locality. In thepreferred embodiment of FIG. 10A, CAD marks 1021, 1031, and 1041 aredisplayed in superposition with the corresponding sub-images 1020, 1030,and 1040, respectively. Preferably, the remainder of the composite image1010 outlying the areas 1020, 1030, and 1040 comprises anon-CAD-specific sub-image 1050 that can consist of information takenfrom an actual two-dimensional mammographic image of the breast, asynthetic two-dimensional mammographic image of the breast derived fromthe received image slices, a single one of the received image slices, ora slabbed image formed by slabbing two or more of the received imageslices.

According to another preferred embodiment (not shown), the localizedregions 1020, 1030, and 1040 can be shown one at a time in the compositeimage 1010, and can be successively invoked using the CAD-basednavigation tools and methods of FIGS. 5, 6, 7A, and 7B supra. It shouldbe noted that it is not necessary to display CAD marks on such asynthesized two dimensional image, but rather, as shown in FIG. 10B, theboundaries of the individual slabbed sub-images would provide a visualindication of the location of any lesion. Whichever mechanism is used,the localized slabbing along cluster boundaries is particularlyadvantageous in increasing the efficiency and accuracy of tomosynthesisimage review.

For one preferred embodiment associated with that of FIGS. 10A-10B, theuser can providing an input indicative of a cine mode request, such asby clicking on the cine control tool 477 of FIGS. 4-5. Responsive tosuch cine mode request, each of the sub-images 1020, 1030, and 1040 arereplaced with a localized cine-loop sequence that displays the imageslices (in individual or variously slabbed subsets) that are spanned bythe associated CAD finding. The individual “mini-cine” sequences areeach laterally windowed according to the localized neighborhood of theassociated CAD finding.

It should be noted that the CAD overlay may be customized according tothe desired work style of the reviewer. It is known that CAD Marks areselected to be displayed to the user by “thresholding”, wherein theoutput of the classifier is a confidence value and any detected regionabove a certain confidence is allocated a mark that displayed to theuser. For this reason, each image may display a different number ofmarks (even though the “average” number is often reported whendescribing the performance of the algorithm), even zero marks in somecases.

According to one aspect of the preferred embodiments, it is realizedthat an alternative method of selecting CAD marks for display is toalways display some constant, designated number of marks per image (e.g.5). This makes the work for the radiologist the same for each case—eachcase requires the review of 5 marks. Sometimes all 5 will be cancers;sometimes all 5 will be false positives. Once the designated number ofmarks are read, if no “cancers” are detected, the radiologist is ensuredthat the remaining portions of the image (whether it be a twodimensional mammogram, tomosynthesis slice or tomosynthesis slab) are beeven “less suspicious”; the chance of there being cancer in otherlocations will be very small and not worth the time to review.Accordingly, this CAD mark display methodology can be used to increasethe efficiency of review of two dimensional and three dimensional imagedata.

FIG. 11 illustrates selecting candidate CAD findings associated withplural x-ray tomosynthesis cases (or plural individual x-raytomosynthesis data volumes) to mark for display according to a preferredembodiment. As known in the art, a typical breast x-ray tomosynthesiscase comprises multiple different views of the left and right breasts ofthe patient, such as left LMO, right MLO, left CC, and right CC views,each view having its own associated x-ray tomosynthesis data volume. Asindicated by the parallel parenthetical expressions below, the method ofFIG. 11 can be carried out on a per case basis, or on a per individualdata volume basis. Generally speaking, the method of FIG. 11 is directedtoward the marking of a predetermined number of candidate CAD findingsper case (or per individual data volume) across all of the cases (orindividual data volumes). The user experience provided is thus weightedmore toward consistency in the number of marked CAD findings per case(or per individual data volume) and less toward uniform evaluation ofthe candidate CAD findings across the different cases (or acrossindividual data volumes).

At step 1102, a target count “N” representing a target number of markedCAD findings to be displayed to a user on a review workstation inconjunction with a plurality of cases (or a plurality of individual datavolumes) is determined, wherein that target count N is independent ofany breast tissue image information contained in any of the cases (orindividual data volumes). By way of example, at the beginning of a dayor at the outset of any particular time interval where many cases willbe reviewed, the radiologist may enter a target count “N” that can be ona per case basis or per data volume basis as desired. Alternatively, thetarget count “N” can be pre-specified according to a stored userprofile, or according to hospital or regulatory standards, and so forth.The target count “N” is independent of any particular breast tissueimage information contained in any of the data volumes that will bepresented to the user.

At step 1104, a next case (or next individual data volume) is received,along with a set of candidate CAD findings associated therewith, eachcandidate CAD finding being associated with a potentially suspiciouslesion as identified by a CAD algorithm and characterizing thepotentially suspicious lesion by a plurality of computed featuresincluding a certainty of finding metric. For each case (or eachindividual data volume), up to the target count N of the candidate CADfindings are designated as being marked CAD findings according to thesteps of, if the number of candidate CAD findings in the received set isless than or equal to the target count N as determined at step 1106,designating all of the candidate CAD findings as marked CAD findings atstep 1108, and if the number of candidate CAD findings is greater thanthe target count N as determined at step 1106, processing the candidateCAD findings according to their computed features, including thecertainty of finding metric, to designate exactly N of the candidate CADfindings as marked CAD findings at step 1110. Finally, at step 1112, thecase data volumes (or the individual data volume) are (is) displayed tothe user on the review workstation with viewable annotation markersthereon corresponding to each of the marked CAD findings, the reviewworkstation not displaying annotation markers corresponding to thecandidate CAD findings that are not marked CAD findings.

When the target count “N” is specified on a per case basis and the caseinvolves multiple data volumes, a variety of different strategies fordistributing the marked CAD findings among the multiple data volumes canbe used and would be apparent to a person skilled in the art in view ofthe instant specification. Thus, for example, if the number of candidateCAD findings in each component data volume of a case is greater than“N”, the designation step 1110 can comprise allocating similar numbersof marked CAD findings to each of the data volumes. Alternatively, thetop “N” CAD findings as determined according to the computed featurescan be selected, regardless of how they are distributed among thecomponent data volumes. As another alternative, if the number ofcandidate CAD findings in each component data volume of a case isgreater than “N”, the designation step 1110 can comprise allocatingsimilar numbers of marked CAD findings to each of the left and rightbreasts.

FIG. 12 illustrates a fibroglandular tissue density map and associatedcertainty-of-finding thresholds for designating candidate CAD findingsas marked CAD findings versus unmarked CAD findings according to apreferred embodiment. Notably, the preferred embodiment of FIG. 12 (aswell as that of FIG. 13 below) are applicable for both two-dimensionalx-ray mammography and breast x-ray tomosynthesis. For the particularcontext of two-dimensional x-ray mammography in a DICOM-based orDICOM-compliant environment, one example of a certainty of findingmetric that can be used in conjunction with the currently describedmethods is the “Certainty of Finding” content item (code schemedesignator DCM, Code Value 111012) from the Mammography CAD StructuredReport Information Object Instance (Mammography CAD SR-IOI) that is usedto describe and communicate the set of CAD findings for a case. Furtherinformation on this content item (111012, DCM, “Certainty of Finding”)can be found in the publication “PS 3.16-2009 Part 16: Content MappingResource,” National Electrical Manufacturers Association (2009), whichis incorporated by reference herein. Similar certainty of findingmetrics exist for breast x-ray tomosynthesis CAD and can be used inconjunction with the described methods. It is to be more generallyappreciated that any CAD-computed data item that is numerically and/orsymbolically representative of a degree of confidence or certainty thata CAD finding is indeed what the CAD algorithm reports it to be can beused as the certainty of finding metric, regardless of the particularnomenclature used to describe that metric.

Notably, as would be readily understood by a person skilled in the art,a CAD-computed certainty of finding metric is different than aCAD-computed probability of malignancy metric, as they representgenerally independent concepts. By way of simplified explanation, a CADalgorithm may identify the presence of a particular mass in the breast,and may characterize that mass as relatively benign (a low probabilityof malignancy metric), but the CAD algorithm may have a very high degreeof certainty about its conclusion that it has indeed found a mass (ahigh certainty of finding metric). By way of further simplifiedexplanation, a CAD algorithm may identify the presence of amicrocalcification cluster with a relatively low degree of certaintythat it is really “looking at” a microcalcification cluster (a lowcertainty of finding metric), but the CAD algorithm may conclude that itis a particularly bad one (high probability of malignancy) if it reallyis a microcalcification cluster.

One example of a fibroglandular tissue density metric that can be usedin conjunction with the presently described preferred embodiments is aso-called “H_(int)” metric described in a book by Ralph Highnam andMichael Brady entitled Mammographic Image Analysis, Kluwer Publishers,Boston Mass. (1999) that describes how to correct and remove the effectsof x-ray scatter, x-ray energy (kVp), exposure (mAs) and breastthickness. See also their PCT Publication WO00/52641A1, which isincorporated by reference herein. The result is a completely physicaldescription of the breast in terms of thickness and type of material—fator fibroglandular tissue. Their interest is in the fibroglandular or“interesting” tissue and thus they call this description H_(int), whichis expressed in units of centimeters, and which represents thecumulative vertical height of fibroglandular tissue above any particularpixel image between the compression plates, the remaining verticalheight representing “non-interesting” tissue, which is primarily fat.Other examples of suitable fibroglandular tissue density metrics arediscussed in Alonzo-Proulx, et. al., “Validation of a Method forMeasuring the Volumetric Breast Density from Digital Mammograms,” Phys.Med. Biol. 55, pp. 3027-3044 (2010), which is incorporated by referenceherein.

According to one preferred embodiment, a computer-implemented method forprocessing and displaying information associated with breast x-rayimages is provided, wherein localized breast fibroglandular tissuedensity information is used together with certainty-of-findinginformation as a basis for selecting which candidate CAD findings todesignate as marked CAD findings. In one preferred embodiment, referringbriefly back to FIG. 1, the original set of candidate CAD findings isgenerated by the CAD processor 112 and provided as one or more DICOM CADstructured reports, whereas the designation of marked versus unmarkedCAD findings is carried out a distinct review workstation 120. Therelevant breast density computations can be carried out be either theCAD processor 112 or the review workstation 120. For one preferredembodiment in which the breast density computations are carried by thereview workstation 120, there is advantageously provided a standalone,segregable capability in which the review workstation 120 can beprogrammed to carry out the methods of FIGS. 12-14 herein and beprovided by a first manufacturer, whereas the CAD processor 112 can be apre-existing device provided by a second equipment manufacturer havingno sense of the methods described herein with respect to FIGS. 12-14.

Preferably, a medical x-ray image of a breast, which can be either atwo-dimensional Mp image or a tomosynthesis data set in differentpreferred embodiments, is received along with a set of candidate CADfindings, each candidate CAD finding identifying a location of apotentially suspicious lesion in the breast and characterizing thepotentially suspicious lesion by a plurality of computed featuresincluding a certainty of finding metric. A fibroglandular tissue densitymap of the breast based on the medical x-ray image is generated. Thefibroglandular tissue density map 1202 characterizes each location inthe medical image by a fibroglandular tissue density metricrepresentative of an absolute proportion, by volume, of fibroglandularbreast tissue in a local neighborhood of that location, with one examplebeing based on the above-described H_(int) metric. Each of the candidateCAD findings is designates as being either a marked CAD finding or anon-marked CAD finding based on its associated certainty of findingmetric and the fibroglandular tissue density metric at the locationthereof. The medical x-ray image is then displayed to a user on a reviewworkstation with viewable annotation markers thereon corresponding toeach of the marked CAD findings, the review workstation not displayingannotation markers corresponding to the non-marked CAD findings.

Preferably, as indicated graphically in FIG. 12 by a fibroglandulardensity legend 1210 placed next to a table 1212 setting forth associatedcertainty of finding thresholds, the designating is carried out suchthat, in order to be designated as marked CAD findings, candidate CADfindings at locations of higher fibroglandular tissue density, such asin region 1206, require higher certainties of finding than is requiredfor candidate CAD findings at locations of lower fibroglandular tissuedensity, such as in region 1208. Especially when programmed on a reviewworkstation functionally and/or commercially segregated from a CADprocessor as described above, a substantial amount of value is added tothe CAD workflow, because it is now “harder” to display a CAD findingwhen that CAD finding has a higher chance of being occluded by highlocalized fibroglandular density in the breast, and “easier” to displaya CAD finding when it is less occluded by localized fibroglandulardensity.

FIG. 13 illustrates an interactive user interface display according to apreferred embodiment that also relates to the fibroglandular tissuedensity map 1202 of FIG. 12. In one preferred embodiment, thefibroglandular tissue density map 1202 is processed to detect acontiguous region of the breast characterized by (i) a fibroglandulartissue density metric that is higher than a predetermined statisticalthreshold, and (ii) a size and shape that is sufficient to substantiallyobscure an anatomical abnormality among the high fibroglandular densitytissue therewithin. Such a size and shape might be, for example, a diskhaving a diameter of 1-2 cm for two-dimensional mammograms, or aspheroid having a diameter of 1-2 cm for tomosynthesis data volumes. Forsuch a detected region, which can be termed a region of excessivefibroglandular tissue density, all of the candidate CAD findings locatedtherein are designated as unmarked CAD findings regardless of their typeor computed features. Preferably, as indicated in FIG. 13, the region ofexcessive fibroglandular tissue density is identified by a highlightedmarking 1397, and a CAD validity warning 1398 is displayed to the userindicating that the area 1397 is too dense for reliable CAD evaluationto occur. Optionally, an alternative modality recommendation 1399 isprovided that recommends an alternative imaging modality for the breast,such as breast ultrasound or breast MRI.

When the medical x-ray image is a two-dimensional x-ray mammogram, ithas been acquired with the breast in a compressed state between twogenerally parallel compression paddles by projecting x-rays through thecompressed breast from an x-ray source positioned on one side of thecompression paddles toward an x-ray detector positioned on an oppositeside of the compression paddles. For these cases, the fibroglandulartissue density map 1202 is a two-dimensional volumetric breast density(VBD) map computed from the two-dimensional x-ray mammogram according toa predetermined VBD computation algorithm, the VBD map containing, foreach location in the x-ray mammogram, information representative of anabsolute cumulative height of the fibroglandular breast tissue and anabsolute cumulative height of non-fibroglandular breast tissue in acorrespondingly located column of breast tissue extending between thecompression paddles.

FIGS. 14A-14B illustrate a conceptual diagram of forming athree-dimensional fibroglandular tissue density map from a plurality ofx-ray tomosynthesis projection images according to a preferredembodiment. When the medical x-ray image is a breast x-ray tomosynthesisvolume, computed from a plurality of x-ray tomosynthesis projectionimages. As illustrated in FIG. 14A, each projection image is acquiredwith the breast B in a compressed state between two generally parallelcompression paddles 1402 and 1404, each projection image being acquiredby projecting x-rays R at a respective tomosynthesis projection angle(such as θ1 and θ2) through the compressed breast from an x-ray sourcepositioned on one side of the compression paddles toward an x-raydetector positioned on an opposite side of the compression paddles, thebreast x-ray tomosynthesis volume comprising a plurality oftwo-dimensional breast x-ray tomosynthesis reconstructed image slicescorresponding to a respective plurality of slice depths in the breastvolume. For these cases, fibroglandular tissue density map 1202 is athree-dimensional map computed from at least two of the x-raytomosynthesis projection images. More specifically, each tomosynthesisprojection image is processed to compute therefrom a respectivetwo-dimensional volumetric breast density (VBD) map according to apredetermined VBD computation algorithm, the VBD map containing, foreach location in the projection image, information representative of anabsolute cumulative height of the fibroglandular breast tissue and anabsolute cumulative height of non-fibroglandular breast tissue in acorrespondingly located column of breast tissue extending between thecompression paddles at the associated tomosynthesis projection angle(see, for example, the voxel columns VCOL(θ1) and VCOL(θ2) in FIG. 14Boverlying a pixel location P). The at least two VBD maps are thenprocessed to generate the three-dimensional fibroglandular tissuedensity map. The three-dimensional fibroglandular tissue density map canbe formed, for example, backprojecting each VBD map intothree-dimensional space and accumulating the results, and/or byprocessing the VBD maps according to a tomosynthesis reconstructionalgorithm.

As illustrated by the conceptual flow diagrams 1412 and 1414 of FIG. 14Brelative to a high-density subvolume HDFG in the breast, the use of suchthree-dimensional fibroglandular tissue density map advantageouslyallows for higher sensitivity for image regions having a lower H_(int)for a greater number of tomosynthesis projection angles. Illustrated inFIG. 14B is a simplified example of the “fibroglandular tissue columns”or “H_(int) columns” that are “seen” by incident radiation at twoprojection angles. The region “A” shown on the resultant fibroglandulardensity map 1416 is highly occluded by virtue of the extended presenceof region HDFG in the voxel column above pixel 1450 at projection angleθ2, and this is not fully “cured” by the information received at pixel1452 at projection angle θ1 because there is some amount of region HDFGin that voxel column as well. A resultant high density map value andassociated high certainty of finding marking threshold is brought aboutfor the region “A”. On the other hand, although the region “B” is alsohighly occluded by virtue of the extended presence of region HDFG in thevoxel column above pixel 1450 at projection angle θ2, this occlusion iscomparatively “cured” by the information received at pixel 1454 atprojection angle θ1 because there is a “clear shot” to that locationfrom angle θ1. The region “B” thereby has a lesser resultant density mapvalue, and a correspondingly lesser certainty of finding markingthreshold, than for the region “A.”

Having described exemplary embodiments, it can be appreciated that theexamples described above are only illustrative and that other examplesalso are encompassed within the scope of the appended claims. Elementsof the system and method are embodied in software; the software modulesof the preferred embodiments have been described to be stored in acomputer readable medium and operable when executed upon by a computerprocessing machine to transform information from two dimensional sliceimages into a displayable representation of the third dimension of thefeature. Several advantages are gained by this transformation; forexample, the time needed to review large sets of image data to detectpotential cancerous lesions can be reduced and the accuracy with which alarge image data set is reviewed is increased. As such, the preferredembodiments fill a critical need in the art to ensure that diagnosticscreening is performed with efficiency and accuracy.

It should also be clear that, as noted above, techniques from knownimage processing and display methods such as post-production of TVimages and picture manipulation by software such as Photoshop fromAdobe, can be used to implement details of the processes describedabove. The above specific embodiments are illustrative, and manyvariations can be introduced on these embodiments without departing fromthe spirit of the disclosure or from the scope of the appended claims.For example, elements and/or features of different illustrativeembodiments may be combined with each other and/or substituted for eachother within the scope of this disclosure.

Whereas many alterations and modifications of the present invention willno doubt become apparent to a person of ordinary skill in the art afterhaving read the foregoing description, it is to be understood that theparticular embodiments shown and described by way of illustration are inno way intended to be considered limiting. By way of example, while thetarget count “N” of marked CAD findings is described above with respectto the preferred embodiment of FIG. 11 supra, as being a single number,at least one alternative preferred embodiment exists in which the targetcount “N” can be a desired range of numbers, wherein at least some ofthe marking decisions can be based on absolute, rather than relative,confidence of finding metrics within any particular case or data volume.Therefore, reference to the details of the preferred embodiments are notintended to limit their scope, which is limited only by the scope of theclaims set forth below.

1. (canceled)
 2. A computer-implemented method for processing anddisplaying breast x-ray tomosynthesis information, the methodcomprising: receiving a plurality of two-dimensional breast x-raytomosynthesis reconstructed image slices of an x-ray tomosynthesisvolume, the image slices corresponding to respective slice depths of aplurality of slice depths in a breast volume; receiving a plurality ofcomputer-aided detection (CAD) findings associated with the breastvolume, each CAD finding identifying a subset of the image slicesspanned by a suspected anatomical abnormality and locations therein ofthe suspected anatomical abnormality; and simultaneously displaying, inan interactive user interface, a first diagnostic image in a firstwindow, a second diagnostic image in a second window, a first graphicaldepth navigation tool configured to graphically communicate to a user animage depth and an image thickness of the first diagnostic image and toallow user control thereof, and a second graphical depth navigation toolconfigured to graphically communicate to the user an image depth and animage thickness of the second diagnostic image and to allow user controlthereof, each of the first graphical depth navigation tool and thesecond graphical depth navigation tool comprising: a slice rulerspatially extending in a first direction representative of the imagedepth, and a slice slider icon disposed along the slice ruler at auser-controllable position corresponding to the image depth, the sliceslider icon comprising a thickness indicator extending along a distanceof the slice ruler corresponding to a thickness of the suspectedanatomical abnormality identified by a CAD finding, and a currentlocation of the slice slider icon and a size of the thickness indicatorbeing automatically adjusted to correspond to a currently selected CADfinding.
 3. The method of claim 2, each of the first diagnostic imageand the second diagnostic image comprising: a single one of the receivedimage slices, or a plurality of depthwise adjacent ones of the receivedimage slices slabbed together.
 4. The method of claim 2, each of thefirst graphical depth navigation tool and the second graphical depthnavigation tool further comprising an extent indication regionidentifying a location and extent of each marked CAD finding of thebreast volume.
 5. The method of claim 4, the extent indication regioncomprising: a plurality of CAD indicator icons corresponding torespective CAD findings and running near and along the slice ruler, eachCAD indicator icon spanning a contiguous segment that corresponds indepthwise position and extent to a subset of image slices spanned by anassociated CAD finding, and at least one single-slice highlighting markpositioned on an associated CAD indicator icon at a location indicativeof the slice depth of a respective one of the subset of image slicesspanned by the associated CAD finding that contains viewable imageinformation corresponding to that associated CAD finding.
 6. The methodof claim 5, wherein the anatomical abnormality identified by a first ofthe CAD findings is a microcalcification cluster, and the at least onesingle-slice highlighting mark on the associated CAD indicator iconidentifies particular ones of the subset of image slices spanned by thatmicrocalcification cluster that contain one or more viewable individualmicrocalcifications in that microcalcification cluster.
 7. The method ofclaim 5, wherein each single-slice highlighting mark is a hash markextending in a direction normal to the first direction and superimposedupon its associated CAD indicator icon.
 8. The method of claim 5,further comprising visually highlighting the CAD indicator icon for thecurrently selected CAD finding to make that CAD indicator icon morenoticeable relative to the other CAD indicator icons on the interactiveuser display.
 9. The method of claim 4, each of the first graphicaldepth navigation tool and the second graphical depth navigation toolcomprising a graphical region of interest (ROI) selector tool positionedin visual proximity to a diagnostic image and a graphical depthnavigation tool, the ROI selector tool allowing the user to individuallyselect CAD findings, and responsive to a user selection at the ROIselector tool, establishing a currently selected CAD finding, modifyingthe displayed diagnostic image to correspond to at least one image slicein the subset of image slices spanned by the currently selected CADfinding.
 10. The method of claim 2, each of the first graphical depthnavigation tool and the second graphical depth navigation toolcomprising a graphical region of interest (ROI) selector tool positionedin visual proximity to a diagnostic image and a graphical depthnavigation tool, the ROI selector tool allowing the user to individuallyselect CAD findings, and responsive to a user selection at the ROIselector tool, establishing a currently selected CAD finding, modifyingthe displayed diagnostic image to correspond to at least one image slicein the subset of image slices spanned by the currently selected CADfinding.
 11. The method of claim 10, the slice slider icon and thethickness indicator being automatically adjusted to correspond to theimage depth and image thickness of the modified diagnostic imageresponsive to the user selection at the ROI selector tool.
 12. Themethod of claim 10, at least one of the first window and the secondwindow comprising a plurality of alternate Region of Interest (ROI) viewwindows.
 13. The method of claim 12, the alternate ROI view windowscomprising: a first alternate ROI view window for CAD findingmagnification, and a second alternate ROI view window comprising agraphical three-dimensional rendering of a CAD finding.
 14. The methodof claim 13, wherein the first and second alternate ROI view windows areautomatically associated with a currently selected CAD finding selectedby the user through the interactive display.
 15. The method of claim 2,the first diagnostic image displayed in the first window comprising aright MLO image, and the second diagnostic image displayed in the secondwindow comprising a left MLO image.
 16. The method of claim 2, at leastone of the first window and the second window comprising anavigation-assisting window comprising markers that provide a condensed,single-view CAD annotation road map for the x-ray tomosynthesis volume.17. The method of claim 2, each of the first graphical depth navigationtool and the second graphical depth navigation tool being configured formulti-dimensional visual indicators related to CAD features, themulti-dimensional visual indicators comprising a size and a shape of aCAD marker corresponding to a lesion attribute.
 18. The method of claim2, each of the first graphical depth navigation tool and the secondgraphical depth navigation comprising an anatomic outlining tool that isactivated by a user to modify pixels of a selected region of adiagnostic image.
 19. The method of claim 18, wherein pixels aremodified based on the user, through the interactive user interface,drawing an outline around a detected feature, and wherein a color of aninterior portion within the drawn outline is modified.
 20. The method ofclaim 2, wherein a displayed diagnostic image includes one or more CADmarkers.
 21. A system configured to process and display breast x-raytomosynthesis information, the system comprising: a data storage device;at least one processing unit in communication with the data storagedevice and an x-ray imager operable to generate tomosynthesis projectionimages, the at least one processing unit being configured or programmedto process the tomosynthesis projection images received from the x-rayimager into an x-ray tomosynthesis volume and to store the x-raytomosynthesis volume to the data storage device, the x-ray tomosynthesisvolume comprising a plurality of two-dimensional breast x-raytomosynthesis reconstructed image slices of a breast volume,two-dimensional breast x-ray tomosynthesis image slices corresponding torespective slice depths of a plurality of slice depths in the breastvolume; and a display unit in communication with the least oneprocessing unit, the at least one processing unit being configured orprogrammed to receive, from the data storage device, the plurality oftwo-dimensional breast x-ray tomosynthesis reconstructed image slices ofthe x-ray tomosynthesis volume and a plurality of computer-aideddetection (CAD) findings associated with the breast volume, each CADfinding identifying a subset of the image slices spanned by a suspectedanatomical abnormality and locations therein of the suspected anatomicalabnormality; and simultaneously display, in an interactive userinterface presented through the display unit, a first diagnostic imagein a first window, a second diagnostic image in a second window, a firstgraphical depth navigation tool configured to graphically communicate toa user an image depth and an image thickness of the first diagnosticimage and to allow user control thereof, and a second graphical depthnavigation tool configured to graphically communicate to the user animage depth and an image thickness of the second diagnostic image and toallow user control thereof, each of the first graphical depth navigationtool and the second graphical depth navigation tool comprising: a sliceruler spatially extending in a first direction representative of theimage depth, and a slice slider icon disposed along the slice ruler at auser-controllable position corresponding to the image depth, the sliceslider icon comprising a thickness indicator extending along a distanceof the slice ruler corresponding to a thickness of the suspectedanatomical abnormality identified by a CAD finding, and a currentlocation of the slice slider icon and a size of the thickness indicatorbeing automatically adjusted to correspond to a currently selected CADfinding.
 22. A non-transitory computer readable medium tangiblyembodying one or more sequences of instructions wherein execution of theone or more sequences of instructions by one or more processorscontained in one or more computing systems causes the one or morecomputing systems to execute a method of processing and displayingbreast x-ray tomosynthesis information, the method comprising: receivinga plurality of two-dimensional breast x-ray tomosynthesis reconstructedimage slices of an x-ray tomosynthesis volume, the image slicescorresponding to respective slice depths of a plurality of slice depthsin a breast volume; receiving a plurality of computer-aided detection(CAD) findings associated with the breast volume, each CAD findingidentifying a subset of the image slices spanned by a suspectedanatomical abnormality and locations therein of the suspected anatomicalabnormality; and simultaneously displaying, in an interactive userinterface, a first diagnostic image in a first window, a seconddiagnostic image in a second window, a first graphical depth navigationtool configured to graphically communicate to a user an image depth andan image thickness of the first diagnostic image and to allow usercontrol thereof, and a second graphical depth navigation tool configuredto graphically communicate to the user an image depth and an imagethickness of the second diagnostic image and to allow user controlthereof, each of the first graphical depth navigation tool and thesecond graphical depth navigation tool comprising: a slice rulerspatially extending in a first direction representative of the imagedepth, and a slice slider icon disposed along the slice ruler at auser-controllable position corresponding to the image depth, the sliceslider icon comprising a thickness indicator extending along a distanceof the slice ruler corresponding to a thickness of the suspectedanatomical abnormality identified by a CAD finding, and a currentlocation of the slice slider icon and a size of the thickness indicatorbeing automatically adjusted to correspond to a currently selected CADfinding.